Endoscope system and image generation method

ABSTRACT

A blue signal B, a green signal G, and a red signal R are obtained by imagining a subject illuminated with white light W using a color CCD  44.  Based on these signals B, G, and R, a normal light image in which a wavelength component of a visible light region is included is generated. Based on the signals B, G, and R, a brightness signal I ((B+G+R)/3) showing the average brightness of the subject is generated. A pixel region exceeding the fixed threshold value Th1 of the brightness signal I is extracted as a superficial microstructure P, such as a pit pattern. A microstructure enhancement image  72  is generated by combining the normal light image with a superficial microstructure image  70  obtained by extracting the superficial microstructure P. The generated microstructure and blood vessel enhancement image  72  is displayed on a monitor  14.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of PCT International Application No.PCT/JP2012/076298 filed on Oct. 11, 2012, which claims priority under 35U.S.C. §119(a) to patent application Ser. No. 2011-225140 filed in Japanon Oct. 12, 2011 and patent application Ser. No. 2012-193907 filed inJapan on Sep. 4, 2012, all of which are hereby expressly incorporated byreference into the present application.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an endoscope system capable of clearlyobserving a blood vessel pattern such as a superficial microvessel, amicrostructure such as a pit pattern, or an irregular pattern such ashypertrophy, and an image generation method.

2. Description of the Related Art

In recent medical treatment, diagnosis or the like using an endoscopeapparatus has been widely performed. In this endoscopic diagnosis, notonly normal light observation, in which white light of broadband lightis used as illumination light within the subject, but also special lightobservation, in which a lesion, such as cancer, is made clearer thanother parts or the position or the size of the lesion is easilyintuitively grasped by using the special light having a specificwavelength as illumination light, is performed.

For example, in JP2001-170009A, using the fact that the degree ofpenetration in the depth direction of the body tissue and the absorptioncharacteristics of hemoglobin in the blood have a wavelength dependency,a microstructure such as a pit pattern or a microvessel formed in a bodytissue surface layer is made clear with blue narrow-band light having ashort wavelength, and a thick blood vessel located in a medium-deeplayer of the body tissue is made clear with green narrow-band lighthaving a longer wavelength than that of the blue narrow-band light.Blood vessels or superficial microstructures of the surface tomedium-deep layers are important clues at the time of differentialdiagnosis of cancer or degree-of-penetration diagnosis. therefore, it ispossible to greatly improve the accuracy of differentiation and the likeby making the blood vessels or the superficial microstructures of thesurface to medium-deep layers clear using blue narrow-band light orgreen narrow-band light.

In addition, in JP1996-252218A (JP-H08-252218A), a boundary between alesion part and a normal part is made clear by using the characteristicthat the amount of auto-fluorescence emitted from the lesion part, whichis thickened due to the lesion such as cancer, is less than the amountof auto-fluorescence from the normal part, which is not thickened, whenirradiating the body tissue with excitation light for exciting theauto-fluorescence. By making the boundary between the lesion part andthe normal part clear as described above, it becomes easy to grasp theposition or the size of the lesion part when performing observation froma distant-view state as at the time of screening.

SUMMARY OF THE INVENTION

In recent years, there are various kinds of cancer differentiationmethods or methods for degree-of-penetration diagnosis. Accordingly,there is not only a case where cancer diagnosis is performed from both ablood vessel pattern, such as a superficial microvessel or a medium-deeplayer blood vessel, and an irregular pattern, such as a superficialmicrostructure or hypertrophy, but also a case where diagnosis isperformed by focusing only on the irregular pattern. For this reason, anendoscope system capable of not only making both a blood vessel patternand an irregular pattern clear but also making only the irregularpattern clear has been requested.

In this regard, according to the method disclosed in JP2001-170009A,both a blood vessel pattern and an irregular pattern can be made clear.However, since a narrow-band wavelength is used in order to make theblood vessel pattern and the irregular pattern clear, the brightness ofthe entire subject is reduced. Therefore, since an image obtained in asituation where the distance from an observation region is great as in adistant-view state becomes a dark image with the reduced amount oflight, the image is not suitable for diagnosis. In addition, accordingto the method disclosed in JP1996-252218A (JP-H08-252218A), it ispossible to grasp the position or the range of the hypertrophy that isone of irregular patterns, but it is difficult to accurately grasp theshape or the size of the irregular pattern since an obtained image is acomplete pseudo-color image.

It is an object of the present invention to provide an endoscope systemand an image generation method capable of obtaining a bright image inwhich a blood vessel pattern or an irregular pattern is clear.

In order to achieve the above-described object, an endoscope system ofthe present invention includes: an image signal acquisition unit foracquiring an image signal by imaging a subject; a normal light imagegeneration unit that generates a normal light image, in which awavelength component of a visible light region is included, based on theimage signal, an irregularity image generation unit that generates anirregularity image by extracting only information of irregularities onthe subject from the image signal; and first composite image generationunit that generates a first composite image by combining the normallight image with the irregularity image.

Preferably, the irregularity image generation unit includes amicrostructure image generation section that generates a microstructureimage as the irregularity image by extracting only a microstructure of abody tissue surface layer from the image signal, and the first compositeimage generation unit includes a normal light image and microstructureimage combination section that generates a first composite image bycombining the normal light image with the microstructure image.Preferably, the irregularity image generation unit includes ahypertrophy image generation section that generates a hypertrophy imageas the irregularity image by extracting from the image signal onlyhypertrophy having a thickness from a body tissue surface layer tomedium-deep layer, and the first composite image generation unitincludes a normal light image and hypertrophy image combination sectionthat generates a first composite image by combining the normal lightimage with the hypertrophy image.

In the present invention, it is preferable to further include: bloodvessel extraction image generation unit that generates a blood vesselextraction image by extracting a blood vessel located at the specificdepth in the subject based on the image signal; and second compositeimage generation unit that generates a second composite image bycombining the normal light image, the blood vessel extraction image, andthe irregularity image. Preferably, the image signal is configured toinclude image signals of a plurality of colors including at least a bluesignal, which has a blue light component having a degree of penetrationto a surface layer of body tissue of the subject, and a green signal,which has a green light component having a degree of penetration to amedium-deep layer of the body tissue, and the blood vessel extractionimage generation unit extracts the blood vessel at the specific depthbased on a ratio of the signal and the green signal of the image signalsof the plurality of colors. Preferably, frequency filtering of afrequency corresponding to a depth of a blood vessel is performed on theblood vessel extraction image.

Preferably, imaging of the subject is performed by a color imagingdevice having pixels of a plurality of colors in which respective colorseparation filters are provided. Preferably, the endoscope system of thepresent invention further includes illumination unit that sequentiallyirradiates the subject with light beams of a plurality of colors, andimaging of the subject is performed by a monochrome imaging devicewhenever sequential irradiation is performed by the illumination unit.Preferably, the endoscope system of the present invention furtherincludes display unit that displays the irregularity image.

An image generation method of the present invention includes: acquiringan image signal by imaging a subject with an imaging device; generatinga normal light image, in which a wavelength component of a visible lightregion is included, based on the image signal using a normal light imagegeneration unit; generating an irregularity image by extracting onlyinformation of irregularities on the subject from the image signal usingirregularity image generation unit; and generating a first compositeimage by combining the normal light image with the irregularity imageusing first composite image generation unit.

According to the present invention, the first composite image obtainedby the first composite image generation unit is an image obtained bycombining the normal light image with the irregularity image obtained byextracting only the information of the irregularities on the subject.Therefore, only an irregular pattern on body tissue, such as asuperficial microstructure or hypertrophy, is made clear in a statewhere the entire brightness is ensured. In addition, the secondcomposite image obtained by the second composite image generation unitis an image obtained by combining the normal light image with theirregularity image and the blood vessel extraction image obtained byextracting the blood vessel at the specific depth. Therefore, anirregular pattern and a blood vessel pattern, such as a superficialmicrovessel, are made clear in a state where the entire brightness isensured.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an endoscope system of a first embodiment.

FIG. 2 is a diagram showing the internal configuration of the endoscopesystem of the first embodiment.

FIG. 3 is a graph showing the emission spectrum of white light W that isexcited and emitted by excitation light.

FIG. 4A is a diagram showing a non-enlarged image captured when the zoomlens is located at wide-angle position.

FIG. 4B is a diagram showing an enlarged image captured when the zoomlens is located at a telephoto position.

FIG. 5 is a graph showing the spectral transmittances of color filtersof R, G, and B colors.

FIG. 6A is a diagram for explaining the imaging control of a CCD in anormal observation mode, a hypertrophy observation mode, and ahypertrophy and blood vessel observation mode in the first embodiment.

FIG. 6B is a diagram for explaining the imaging control of the CCD in asuperficial microstructure observation mode and a microstructure andblood vessel observation mode in the first embodiment.

FIG. 7 is a diagram showing the internal configuration of a speciallight image generation unit.

FIG. 8 is a diagram for explaining a method of generating a superficialmicrostructure image.

FIG. 9 is a diagram showing a superficial microstructure enhancementimage.

FIG. 10 is a diagram for explaining a method of generating a hypertrophyimage.

FIG. 11 is a diagram showing hypertrophy, a part that is nothypertrophy, and reflected light R1, R2, and R3 from the medium-deeplayer blood vessel.

FIG. 12 is a diagram showing a hypertrophy enhancement image.

FIG. 13 is a graph showing the relationship between the brightness ratioB/G and the blood vessel depth.

FIG. 14 is a graph showing the brightness distribution of a part of aB/G image.

FIG. 15 is a graph showing the brightness distribution of a part of asuperficial blood vessel extraction image.

FIG. 16 is a graph showing the brightness distribution of a part of amedium-deep layer blood vessel image.

FIG. 17 is a diagram showing a microstructure and blood vesselenhancement image.

FIG. 18 is a diagram showing a hypertrophy and blood vessel enhancementimage.

FIG. 19 is a flowchart showing a sequential flow in the superficialmicrostructure observation mode or the microstructure and blood vesselobservation mode.

FIG. 20 is a flowchart showing a sequential flow in the hypertrophyobservation mode or the hypertrophy and blood vessel observation mode.

FIG. 21 is a diagram showing the internal configuration of an endoscopesystem of a second embodiment.

FIG. 22 is a diagram showing a rotary filer.

FIG. 23 is a graph showing the spectral transmittances of a B filter, aG filter, and an R filter and the emission intensities of broadbandlight BB and blue narrow-band light RN.

FIG. 24A is a diagram for explaining the imaging control of a CCD in anormal observation mode, a hypertrophy observation mode, and ahypertrophy and blood vessel observation mode in the second embodiment.

FIG. 24B is a diagram for explaining the imaging control of the CCD in asuperficial microstructure observation mode and a microstructure andblood vessel observation mode in the second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

As shown in FIGS. 1 and 2, en endoscope system 10 of the firstembodiment includes: an electronic endoscope 11 (a form of image signalacquisition unit) that images the inside of a subject, a processordevice 12 that performs various kinds of image processing on the imagecaptured by the electronic endoscope 11; a light source device 13 thatsupplies light for illuminating the subject to the electronic endoscope11; and a monitor 14 that displays an image after various kinds of imageprocessing are performed by the processor device 12.

The electronic endoscope 11 includes a flexible insertion unit 16 thatis inserted into the subject, an operating unit 17 provided at theproximal end of the insertion unit 16, and a universal code 18 thatmakes a connection between the operating unit 17 and the processordevice 12 and the light source device 13. A curved portion 19 obtainedby connecting a plurality of curved pieces is formed at the distal endof the insertion unit 16. The curved portion 19 is curved in thehorizontal and vertical directions by operating an angle knob 21 of theoperating unit 17. A distal portion 16 a including an optical system forimaging the body cavity and the like is provided at the distal end ofthe curved portion 19. The distal portion 16 a is directed in a desireddirection within the subject by the bending operation of the curvedportion 19.

In addition, a mode switch SW 15 for switching to various modes isprovided in the operating unit 17. The various modes include a total offive modes of a normal observation mode in which a normal light imageobtained by imaging a subject illuminated with white light is displayedon the monitor 17, a superficial microstructure observation mode inwhich a superficial microstructure enhancement image emphasizing themicrostructure formed on the surface layer of body tissue is displayedon the monitor 14, a hypertrophy observation mode in which a hypertrophyenhancement image that emphasizes a hypertrophy having a thickness fromthe surface layer to the medium-deep layer in body tissue is displayedon the monitor 14, a microstructure and blood vessel observation mode inwhich a microstructure and blood vessel enhancement image emphasizingthe superficial microstructure and the blood vessel located at aspecific depth is displayed on the monitor 14, and a hypertrophy andblood vessel observation mode in which a hypertrophy and blood vesselenhancement image emphasizing hypertrophy and the blood vessel locatedat a specific depth is displayed on the monitor 14.

A connector 24 is attached to the universal code 18 on the side of theprocessor device 12 and the light source device 13. The connector 24 isa composite connector including a communication connector and a lightsource connector, and the electronic endoscope 11 is detachablyconnected to the processor device 12 and the light source device 13through the connector 24.

The light source device 13 includes: an excitation light source 30 thatemits excitation light EL having a specific wavelength, a bluenarrow-band light source 31 that emits blue narrow-band light BN that isnarrowed to a specific wavelength in the blue band, an optical fiber forexcitation light 35 on which the excitation light EL from the excitationlight source 30 is incident; and optical fiber for blue narrow-bandlight 36 on which the blue narrow-band light BN from the bluenarrow-band light source 31 is incident; a coupler 37 that opticallycouples the optical fibers 35 and 36 with a light guide 43 in theelectronic endoscope; and a light source switching unit 39 for ON/OFFswitching of the excitation light source 30 and the blue narrow-bandlight source 31.

The excitation light source 30 is a semiconductor light source, such asa laser diode, and emits the excitation light EL having a centerwavelength of 445 nm as shown in FIG. 3. The excitation light EL isirradiated to a phosphor 40, which is provided in the distal portion 16a of the electronic endoscope 11, through the optical fiber forexcitation light 35, the coupler 37, and the light guide 43. Thephosphor 40 is configured to include a plurality of kinds of fluorescentmaterials (for example, a YAG-based fluorescent material or afluorescent material, such as BAM (BaMgAl₁₉O₁₇) that absorb a part ofthe excitation light EL and excite and emit fluorescence FL of green tored. As shown in FIG. 3, the fluorescence FL excited and emitted by thephosphor 40 is combined with the excitation light EL that is transmittedthrough the phosphor 40 without being absorbed by the phosphor 40,thereby generating white light W.

The blue narrow-band light source 31 is a semiconductor light source,such as a laser diode, and emit the blue narrow-band light BN having acenter wavelength of 405 nm as shown in FIG. 3. Since the bluenarrow-band light BN has a degree of penetration to the surface layer ofbody tissue, the blue narrow-band light BN is used in order to brightlyilluminate the superficial microstructure of body tissue, such as a pitpattern.

The light source switching unit 39 is connected to a controller 59 inthe processor device, and switches the excitation light source 30 andthe blue narrow-band light source 31 ON (lighting) and OFF (lightingoff) according to the mode that is set. When the normal observationmode, the hypertrophy observation mode, and the hypertrophy and bloodvessel observation mode are set, the excitation light source 30 isalways ON while the blue narrow-band light source 31 is always OFF.Accordingly, only the white light W is always irradiated to the subject.In contrast, when the superficial microstructure observation mode or themicrostructure and blood vessel observation mode is set, the excitationlight source 30 is always ON while the blue narrow-band light source 31is switched ON and OFF in every other frame. Accordingly, irradiation ofonly the white light W and simultaneous irradiation of the white light Wand the blue narrow-band light BN are alternately repeated every otherframe.

The electronic endoscope 11 includes the light guide 43, a CCD 44, ananalog processing circuit 45 (AFE: Analog Font End), an imaging controlunit 46, and a magnification control unit 47. The light guide 43 is alarge-diameter optical fiber, a bundle fiber, or the like, and theincidence end is inserted in the coupler 37 of the light source deviceand the exit end is directed to the phosphor 40. Light guided by thelight guide 43 is irradiated to the subject through the phosphor 40, anirradiation lens 48 b, and an illumination window 49. An observationwindow 50 receives returned light from the subject. The received lightis incident on the CCD 44 through a condensing lens 51 and a zoom lens48 a.

An actuator 48 c to move the zoom lens 48 a in the optical axisdirection is attached to the zoom lens 48 a. The driving of the actuator48 c is controlled by the magnification control unit 47 connected to thecontroller 59. The magnification control unit 47 controls the actuator48 c so that the zoom lens 48 a moves to a position corresponding to themagnification set by a zoom operation unit 20. When it is necessary toobserve the overall condition in the subject, for example, at the timeof screening, the zoom lens 48 a is set to a wide-angle position so thata non-enlarged image shown in FIG. 4A is displayed on the monitor 14. Onthe other hand, when it is necessary to observe the detailed structureof a part to be observed, for example, at the time of differentialdiagnosis of cancer, the zoom lens 48 a is set to a telephoto positionso that an enlarged image shown in FIG. 4B is displayed on the monitor14.

In the normal observation mode, the hypertrophy observation mode, andthe hypertrophy and blood vessel observation mode, the overall conditionin the subject is observed in many cases. Therefore, the zoom lens 48 ais set to the wide-angle position in many cases. On the other hand, inthe superficial microstructure observation mode and the microstructureand blood vessel observation mode, an object to be observed is enlargedand observed in many cases. Therefore, the zoom lens 48 a is set to thetelephoto position in many cases.

The CCD 44 has an imaging surface 44 a on which light from thecondensing lens 51 is incident, and performs photoelectric conversion ofthe light received by the imaging surface 44 a and accumulates thesignal charges. The accumulated signal charges are read as an imagingsignal, and the imaging signal is transmitted to the AFE 45. The CCD 44is a color CCD, and pixels of three colors of a B pixel in which a colorfilter of B color is provided, a G pixel in which a color filter of Gcolor is provided, and an R pixel in which a color filter of R color isprovided are arrayed on the imaging surface 44 a. These color filters ofB, G, and R colors have spectral transmittances indicated by curves 52,53, and 54 shown in FIG. 5.

The AFE 45 is configured to include a correlated double sampling circuit(CDS), an automatic gain control circuit (ARC), and an analog/digitalconverter (A/D) (all not shown). The CDS performs correlated doublesampling processing on an imaging signal from the CCD 44 to remove noisecaused by the driving of CCD 44. The AGC amplifies an imaging signalfrom which noise has been removed by the CDS. The A/D converts animaging signal amplified by the AGC into a digital imaging signal of apredetermined number of bits, and inputs the digital imagining signal tothe processor device 12.

The imaging control unit 46 is connected to the controller 59 in theprocessor device 12, and transmits a driving signal to the CCD 44 whenthere is an instruction from the controller 59. The CCD 44 outputs animaging signal to the AFE 45 at a predetermined frame rate based on thedriving signal from the imaging control unit 46.

In the normal observation mode, the hypertrophy observation mode, andthe hypertrophy and blood vessel observation mode, as shown in FIG. 6A,a step of performing photoelectric conversion of image light of thewhite light W and accumulating the signal charges and a step of readingthe accumulated signal charges are performed within one frame period.This imaging control is repeatedly performed while the normalobservation mode, the hypertrophy observation mode, and the hypertrophyand blood vessel observation mode are set. In each read step, a bluesignal B, a green signal G, and a red signal R are respectively outputfrom the B, G, and R pixels of the CCD 44.

When the superficial microstructure observation mode or themicrostructure and blood vessel observation mode is set, as shown inFIG. 6B, a step of performing photoelectric conversion of image light ofthe white light W and accumulating the signal charges and a step ofreading the accumulated signal charges are performed within one frameperiod. Then, a step of performing photoelectric conversion of imagelight of the white light W and the blue narrow-band light BN andaccumulating the signal charges and a step of reading the accumulatedsignal charges are performed within one frame period. This imagingcontrol of two frames is repeatedly performed while the surface layerobservation mode is set. In each read step, a blue signal B1, a greensignal G1, and a red signal R1 are respectively output from the B, G,and R pixels of the CCD 44 in the first frame, and a blue signal B2, agreen signal G2, and a red signal R2 are respectively output from the B,G, and R pixels in the next, second frame.

As shown in FIG. 2, the processor device 12 includes a normal lightimage generation unit 55 (a form of normal light image generation unit),a frame memory 56, a special light image generation unit 57 (a form ofirregularity image generation unit), and a display control circuit 58.The controller 59 controls each of the units. The normal light imagegeneration unit 55 generates a normal light image from a signal obtainedby imaging the image light of the white light W with the electronicendoscope 11. That is, in the normal observation mode, the hypertrophyobservation mode, and the hypertrophy and blood vessel observation mode,a normal light image is generated from the blue signal B, the greensignal G, and the red signal R. In the superficial microstructureobservation mode or the microstructure and blood vessel observationmode, a normal light image is generated from the blue signal B1, thegreen signal G1, and the red signal R1. The generated normal light imageis temporarily stored in the frame memory 56.

As shown in FIG. 7, the special light image generation unit 57 includesa superficial microstructure enhancement image generation section 61, ahypertrophy enhancement image generation section 62, and anirregularities and blood vessel enhancement image generation section 63.The superficial microstructure enhancement image generation section 61includes a brightness signal generation section 61 a that generates abrightness signal I indicating the average brightness of the subject, amicrostructure extraction section 61 b (microstructure image generationsection) that extracts a superficial microstructure image from thebrightness signal I, and an image generation section 61 c (normal lightimage and microstructure image combination section that is firstcomposite image generation unit) that generates a superficialmicrostructure enhancement image by combining the normal light imagewith the superficial microstructure image extracted by themicrostructure extraction section 61 b.

The brightness signal generation section 61 a generates the brightnesssignal I (=(B2+G2+R2)/3) by dividing the total value (B2+G2+R2) of thesignal values of the signals B2, G2, and R2 of the second frame, whichare obtained in the superficial microstructure observation mode or themicrostructure and blood vessel observation mode, by 3. Since thebrightness signal I is obtained by averaging the signal values of therespective colors, the brightness signal I indicates the averagebrightness of the subject. As shown in FIG. 8(A), the microstructureextraction section 61 b extracts a pixel region, which exceeds a fixedhigh-brightness threshold value Th1 of the brightness signal I, as asuperficial microstructure P. As shown in FIG. 8(B), in a superficialmicrostructure image in which the superficial microstructure P isextracted, the pixel value of the brightness signal I is used as it isfor a pixel region exceeding the high-brightness threshold value Th1,while the pixel value is set to “0” for a pixel region equal to or lessthan the high-brightness threshold value Th1.

The reason why the pixel region exceeding the high-brightness thresholdvalue Th1 is set as the superficial microstructure P as described aboveis as follows. The superficial microstructure is formed of a pit patternin which micropores formed on the body tissue surface layer gather, forexample. Accordingly, when white light or the blue narrow-band light BNis incident on the micropores of the pit pattern, a multiple scatteringphenomenon occurs so that the micropores shine brightly. Therefore, abright region exceeding the high-brightness threshold value Th1 is setas a superficial microstructure. In addition, the superficialmicrovessel V1 is present between gland ducts in many cases. Since thesuperficial microvessel shows the strong absorption characteristics forthe blue component or the blue narrow-band light BN of white light, thesuperficial microvessel is displayed darkly on the image. Accordingly,since the brightness of the superficial microvessel V1 is almost lessthan the high-brightness threshold value Th, superficial microvesselsare hardly present (hardly included) in the superficial microstructureimage.

Then, high-frequency filtering processing is performed on thesuperficial microstructure image. The brightness value of a partcorresponding to the superficial microstructure P changes greatlycompared with other parts. Therefore, as shown in FIG. 8(C), asuperficial microstructure image 70 in which only the superficialmicrostructure P is sharply extracted is obtained by performinghigh-frequency filtering.

The image generation section 61 c generates a superficial microstructureenhancement image 72 shown in FIG. 9 by combining the normal light imagewith the superficial microstructure image 70 obtained by themicrostructure extraction section 61 b. The generated superficialmicrostructure enhancement image 72 is displayed on the monitor 14 bythe display control circuit 58. Since the superficial microstructureenhancement 72 is based on the bright normal image, it is possible toperform diagnosis while securing the visibility. In the superficialmicrostructure enhancement image 72, only the superficial microstructureP of the body tissue of the surface layer other than a superficialmicrovessel V1 is clear. Accordingly, the superficial microstructureenhancement image 72 is an image that is suitable when performing cancerdiagnosis by focusing on only the superficial microstructure P.

As shown in FIG. 7, the hypertrophy enhancement image generation section62 includes: a gain adjustment section 62 a that adjusts the gain of thesignal B, G, and R in order to increase a signal including theinformation of a hypertrophy portion; a brightness signal generationsection 62 b that generates the brightness signal I indicating theaverage brightness of the subject from the signals B, G, and R aftergain adjustment; a hypertrophy extraction section 62 c (hypertrophyimage generation section) that extracts a hypertrophy image from thebrightness signal I; and an image generation section 62 d (normal lightimage and hypertrophy image combination section that is first compositeimage generation unit) that generates a hypertrophy enhancement image bycombining the normal light image with the hypertrophy image extracted bythe hypertrophy extraction section.

The gain adjustment section 62 a multiplies a pixel value by the gain of1 of less for the signal B of the signals B, G, and R obtained in thehypertrophy observation mode or the hypertrophy and blood vesselobservation mode, and multiplies a pixel value by the gain of 1 or morefor the signals G and R. Since the hypertrophy has a thickness from thesurface layer to the medium-deep layer, a lot of information regardingthe hypertrophy is included not in the reflected light of light having ashort wavelength less than 500 nm but in the reflected light of lighthaving a long wavelength exceeding 500 nm. Therefore, the informationregarding the hypertrophy can be made to be noticeable by reducing thepixel value of the signal B including a lot of information of thereflected light of light having a wavelength less than 500 nm andincreasing the pixel values of the signals G and R including a lot ofinformation of the reflected light of light having a wavelengthexceeding 500 nm.

The brightness signal generation section 62 b generates the brightnesssignal I (=(B+G+R)/3) by dividing the total value (B+G+R) of the signalvalues of the signals B, G, and R after gain adjustment by 3. Since thebrightness signal I is obtained by averaging the signal values of therespective colors, the brightness signal I indicates the averagebrightness of the subject. As shown in FIG. 10(A), the hypertrophyextraction section 62 c extracts a pixel region, in which the pixelvalue is within a medium brightness range M of threshold values Th2 toTh3, of the brightness signal I as hypertrophy Q. As shown in FIG.10(B), in a hypertrophy image 80 in which the hypertrophy Q isextracted, the pixel value is used as it is for a pixel region where thepixel value is within the medium brightness range M, while the pixelvalue is set to “0” for a pixel region where the pixel value is notwithin the medium brightness range M.

The reason why the pixel region where the pixel value is within themedium brightness range M is set as the hypertrophy Q is as follows. Asshown in FIG. 11, in the hypertrophy Q, the thickness from the surfacelayer to the medium-deep layer is increased due to a ridge from the bodytissue surface layer, compared with other parts that are not thickened.Accordingly, when emitting light having a wavelength exceeding 500 nmthat has a degree of penetration to the medium-deep layer, the amount ofreflected light R1 from the hypertrophy Q is reduced compared with theamount of reflected light R2 of other parts that are not thickened. Onthe other hand, the amount of reflected light R1 from the hypertrophy Qis not reduced compared with the amount of reflected light R3 whenemitting light having a wavelength exceeding 500 nm to a medium-deeplayer blood vessel V2. This is because the absorbance of the mucousmembrane that forms the hypertrophy Q is lower than the absorbance ofhemoglobin in the blood vessel. Therefore, a pixel region where thepixel value is within the medium brightness range M is set as thehypertrophy Q.

Then, low-frequency to medium-frequency filtering processing isperformed on the hypertrophy image 80. A change in the brightness valueof the hypertrophy portion is small compared with other parts.Therefore, as shown in FIG. 10(C), the hypertrophy image 80 in whichonly the hypertrophy is sharply extracted is obtained by performinglow-frequency to medium-frequency filtering.

The image generation section 62 d generates a hypertrophy enhancementimage 82 shown in FIG. 12 by combining the normal light image with thehypertrophy image 80 obtained by the hypertrophy extraction section 62c. The generated hypertrophy enhancement image 82 is displayed on themonitor 14 by the display control circuit 58. Since the hypertrophyenhancement image 82 is based on the bright normal light image, it ispossible to perform diagnosis while securing the visibility. In thehypertrophy enhancement image 82, only the hypertrophy Q of thebiological information of the body tissue surface layer is clear.Accordingly, the hypertrophy enhancement image 82 is an image that issuitable when performing cancer diagnosis by focusing on only thehypertrophy.

As shown in FIG. 7, the irregularities and blood vessel enhancementimage generation section 63 (a form of second composite image generationunit) includes a B/G image generation section 63 a, a blood vesselextraction section 63 b (a form of blood vessel extraction imagegeneration unit), and an irregularities and blood vessel enhancementimage generation section 63 c. The B/G image generation section 63 agenerates a B/G image from the signal of the blue component and thesignal of the green component among the signals acquired by the imagingof the image light of the white light. In each pixel of the B/G image,information of the brightness ratio B/G obtained by dividing the signalvalue of the blue signal B by the signal value of the green signal G isincluded. In the hypertrophy observation mode or the hypertrophy andblood vessel observation mode, the B/G image is generated from B and G.On the other hand, in the superficial microstructure observation mode orthe microstructure and blood vessel observation mode, the B/G image isgenerated from B1 and G1.

The brightness ratio B/G of each pixel in the B/G image is relevant tothe blood vessel depth. As shown in FIG. 13, there is a proportionalrelationship in which the brightness ratio B/G increases as the bloodvessel depth increases. Accordingly, the magnitude relationship of“brightness of superficial blood vessel <brightness of mucous membrane<brightness of medium-deep layer blood vessel” is satisfied. Inaddition, as shown in the B/G image of FIG. 14, due to factors such asuneven brightness, there is a case where the brightness of the B/G imageis the highest in the middle and decreases toward the periphery from thecenter. Therefore, the above-described magnitude relationship(brightness of superficial blood vessel <brightness of mucous membrane<brightness of medium-deep layer blood vessel) is locally satisfied, butmay not be satisfied on the whole.

The blood vessel extraction section 63 b generates a blood vesselextraction image by extracting a blood vessel located in a specificlayer from the B/G image. The blood vessel extraction is performed byfrequency filtering processing. When extracting a superficial bloodvessel, high-frequency components, which are frequency band components,many of which are present in the superficial blood vessel, are extractedfrom the B/G image. As a result, as shown in FIG. 15, it is possible toobtain a superficial blood vessel extraction image in which thebrightness of the superficial blood vessel is negative and thebrightness of a mucous membrane portion is approximately “0”. In thesuperficial blood vessel extraction image, only the superficial bloodvessel is extracted sharply. On the other hand, when extracting amedium-deep layer blood vessel, medium-frequency components, which arefrequency band components, many of which are present in the medium-deeplayer blood vessel, are extracted from the B/G image. As a result, asshown in FIG. 16, it is possible to obtain a medium-deep layer bloodvessel extraction image in which the brightness of the medium-deep layerblood vessel is positive and the brightness of a mucous membrane portionis approximately “0”. In the medium-deep layer blood vessel extractionimage, only the medium-deep layer blood vessel is extracted sharply.

Since the component of the mucous membrane becomes a value close toapproximately “0” by performing the frequency filtering processingdescribed above, it is possible to extract only a blood vessel portion.In addition, the above-described magnitude relationship (brightness ofsuperficial blood vessel <brightness of mucous membrane <brightness ofmedium-deep layer blood vessel) is also satisfied on the whole.

When the microstructure and blood vessel observation mode is set, theirregularities and blood vessel enhancement image generation section 65generates a microstructure and blood vessel enhancement image, in whichthe superficial microstructure and the blood vessel of a specific layerare clear, by combining the superficial microstructure image, the bloodvessel extraction image, and the normal light image. The generatedmicrostructure and blood vessel enhancement image is displayed on themonitor 14 by the display control circuit 58. For example, when asuperficial blood vessel extraction image is used as a blood vesselextraction image, a microstructure and blood vessel enhancement image 90in which both a pit pattern P, which is configured to include a numberof gland ducts, and a microvessel V1 between the gland ducts are clearis displayed, as shown in FIG. 17. Thus, by making clear both the pitpattern P and the microvessel V1, it is possible to improve thediagnostic performance in differentiating cancer.

On the other hand, when the hypertrophy and blood vessel observationmode is set, the irregularities and blood vessel enhancement imagegeneration section 65 generates a hypertrophy and blood vesselenhancement image, in which hypertrophy and the blood vessel of aspecific layer are clear, by combining the hypertrophy image, the bloodvessel extraction image, and the normal light image. The generatedhypertrophy and blood vessel enhancement image is displayed on themonitor 14 by the display control circuit 58. For example, when amedium-deep layer blood vessel extraction image is used as a bloodvessel extraction image, a hypertrophy and blood vessel enhancementimage 92 in which both the hypertrophy Q and the medium-deep layer bloodvessel V2 are clear is displayed, as shown in FIG. 18. Thus, by makingclear both the medium-deep layer blood vessel V2 and the hypertrophy Qwith a possibility of cancer, it becomes easy to detect a lesion at thetime of screening.

Next, a sequential flow in the superficial microstructure observationmode and the microstructure and blood vessel observation mode will bedescribed with reference to the flowchart shown in FIG. 19.

When switching to the superficial microstructure observation mode or themicrostructure and blood vessel observation mode is performed by themode switch SW 15, the white light W is irradiated toward the subject.By imaging the subject illuminated with the white light W using thecolor CCD 44, the blue signal B1, the green signal G1, and the redsignal R1 are output from the CCD 44. Then, the white light W and theblue narrow-band light BN are simultaneously irradiated toward thesubject. By imaging the subject illuminated with the white light W andthe blue narrow-band light BN using the CCD 44, the blue signal B2, thegreen signal G2, and the red signal R2 are output from the CCD 44.

Then, a normal light image is generated based on the blue signal B1, thegreen signal G1, and the red signal R1. Then, a brightness signal I isgenerated by dividing the total value (B2+G2+R2) of the signal values ofthe blue signal B2, G2, and R2 by 3. A superficial microstructure imageis generated by extracting a pixel region exceeding the high-brightnessthreshold value Th1 of the brightness signal I. By performinghigh-frequency filtering on the generated superficial microstructureimage 70, the superficial microstructure in the image is sharpened.

When the superficial microstructure observation mode is set, asuperficial microstructure enhancement image is generated by combiningthe normal light image with the superficial microstructure image afterfrequency filtering. The generated superficial microstructureenhancement image is displayed on the monitor 14. The series ofoperations described above are repeatedly performed while thesuperficial microstructure observation mode is set.

On the other hand, when the microstructure and blood vessel observationmode is set, a B/G image having a brightness ratio B/G between the bluesignal B1 and the green signal G1 is generated. After the B/G image isgenerated, a blood vessel of a specific layer is extracted from the B/Gimage. As a result, a blood vessel extraction image is obtained. Amicrostructure and blood vessel enhancement image is generated bycombining the blood vessel extraction image, the superficialmicrostructure enhancement image 70, and the normal light image. Thegenerated microstructure and blood vessel enhancement image is displayedon the monitor 14. The series of operations described above arerepeatedly performed while the microstructure and blood vesselobservation mode is set.

Next, a sequential flow in the hypertrophy observation mode and thehypertrophy and blood vessel observation mode will be described withreference to the flowchart shown in FIG. 19.

When switching to the hypertrophy observation mode or the hypertrophyand blood vessel observation mode is performed by the mode switch SW 15,the white light W is irradiated toward the subject. By imaging thesubject illuminated with the white light W using the color CCD 44, theblue signal B, the green signal G, and the red signal R are output fromthe CCD 44.

Then, a normal light image is generated based on the blue signal B, thegreen signal G, and the red signal R. In addition, the gain of thesignals B, G, and R is adjusted to reduce the pixel value of the signalB and increase the pixel values of the signals G and R. Then, thebrightness signal I is generated by dividing the total value (B+G+R) ofthe signal values of the signals B, G, and r after gain adjustment by 3.A hypertrophy image is generated by extracting a pixel region less thanthe low-brightness threshold value Th2 of the brightness signal I. Byperforming low-frequency to medium-frequency filtering on the generatedhypertrophy image, the hypertrophy in the image is sharpened.

When the hypertrophy observation mode is set, a hypertrophy enhancementimage is generated by combining the normal light image with thehypertrophy image after frequency filtering. The generated hypertrophyenhancement image is displayed on the monitor 14. The series ofoperations described above are repeatedly performed while thehypertrophy observation mode is set.

On the other hand, when the hypertrophy and blood vessel observationmode is set, a B/G image having a brightness ratio B/G between the bluesignal B and the green signal G is generated. After the B/G image isgenerated, a blood vessel of a specific layer is extracted from the B/Gimage. As a result, a blood vessel extraction image is obtained. Ahypertrophy and blood vessel enhancement image is generated by combiningthe blood vessel extraction image, the hypertrophy enhancement image,and the normal light image. The generated hypertrophy and blood vesselenhancement image is displayed on the monitor 14. The series ofoperations described above are repeatedly performed while thehypertrophy and blood vessel observation mode is set.

In a second embodiment of the present invention, unlike the firstembodiment of the simultaneous system using the color CCD, subjectimaging is performed in a frame sequential method using the rotaryfilter of RGB. As shown in FIG. 21, in an endoscope system 100 of thesecond embodiment, the configuration of a light source device 113 isdifferent from that of the light source device 13 of the firstembodiment. In addition, the phosphor 40 is not provided in the distalportion 16 a of the electronic endoscope 11. In addition, theconfiguration of the CCD and the operation of the imaging control unit46 in the electronic endoscope 11 are different from those in the firstembodiment. In addition, the method of generating a normal light imagein the microstructure observation mode and the microstructure and bloodvessel observation mode is different. Since others are the same as thosedescribed in the first embodiment, only the differences from the firstembodiment will be described below.

The light source device 113 includes: a white light source 130 thatemits broadband light BB 400 nm to 700 nm); a rotary filter 134 thatseparates the broadband light BB from the white light source 130 intolight beams of three colors of B, G, and R and sequentially suppliesthese light beams of respective colors to the light guide 43; asemiconductor light source unit 136 that emits the blue narrow-bandlight BN; and a light joining unit 138 that makes the blue narrow-bandlight BN join the optical path L1 of the broadband light BB between therotary filter 134 and the light guide 43.

The white light source 130 is formed of a xenon lamp, a halogen lamp, ametal halide, or the like. As shown in FIG. 22, the rotary filter 134 isrotatably provided so that a B filter portion 134 a, a G filter portion134 b, and an R filter portion 134 c are selectively inserted in theoptical path L1 of the broadband light BB. The rotary filter 134 has adisc shape, and is divided into three regions, each of which is afan-shaped region having a central angle of 120°, in the circumferentialdirection, and the B filter portion 134 a, the G filter portion 134 b,and the R filter portion 134 c are respectively provided in the threeregions.

As shown in FIG. 23, the B filter portion 134 a allows B light of theblue band from the broadband light BB to be transmitted therethrough,the G filter portion 134 b allows G light of the green band from thebroadband light BB to be transmitted therethrough, and the R filterportion 134 c allows R light of the red band from the broadband light BBto be transmitted therethrough. Therefore, B light, G light, and R lightare sequentially emitted from the rotary filter 134 by the rotation ofthe rotary filter 134.

The semiconductor light source unit 136 includes a blue narrow-bandlight source 136 a, an optical fiber 136 b, and a light source controlsection 136 c. As shown in FIG 23, the blue narrow-band light source 136a is a semiconductor light source, such as a laser diode, and emits theblue narrow band BN having a center wavelength of 405 nm. The bluenarrow-band light source 136 a is turned on and off according to thecontrol of the light source control section 136 c. In the normalobservation mode, the hypertrophy observation mode, and the hypertrophyand blood vessel observation mode, the blue narrow-band light source 236a is always in the OFF state. In the microstructure observation mode andthe microstructure and blood vessel observation mode, only the bluenarrow-band light source 136 a is turned on only when the B filter 134 aof the rotary filter 134 is inserted in the optical path L1 of thebroadband light BB, and the blue narrow-band light source 136 a isturned off when the other G and R filters 134 b and 134 c are insertedin the optical path L1.

The blue narrow-band light BN from the blue narrow-band light source 136a is incident on the optical fiber 136 b. The blue narrow-band light BNis emitted toward the light joining unit 138 through a collimator lens(not shown). The light joining unit 138 is a dichroic mirror. Light fromthe rotary filter 134 is transmitted through the light joining unit 138as it is, while the blue narrow-band light BN from the semiconductorlight source unit 136 is reflected from the light joining unit 138.Accordingly, the optical path L2 of the blue narrow-band light BN ismade to match the optical path L1 of the broadband light BB. Lightemitted from the light joining unit 138 is incident on the light guide43 through a condensing lens 142.

A CCD 144 in the electronic endoscope is a monochrome imaging device inwhich a micro color filter is not provided on the imaging surface,unlike in the first embodiment described above. The imaging control unit46 that controls the imaging of the CCD 144 also performs a differentoperation from that in the first embodiment described above.

In the normal observation mode, the hypertrophy observation mode, andthe hypertrophy and blood vessel observation mode, as shown in FIG. 24A,image light beams of three colors of B, G, and R are sequentiallycaptured and the electric charges are accumulated, and frame sequentialimaging signals B, G, and R are sequentially output based on theaccumulated electric charges. The series of operations are repeatedwhile the normal observation mode, the hypertrophy observation mode, andthe hypertrophy and blood vessel observation mode are set. The framesequential imaging signals B, G, and R correspond to B, G, and R of thefirst embodiment, respectively.

In the microstructure observation mode and the microstructure and bloodvessel observation mode, as shown in FIG. 24B, image light of mixedcolor light of the light of B color and the blue narrow-band light BNthat are simultaneously irradiated, image light of the light of G color,and image light of the light of R color are sequentially captured andthe electric charges are accumulated, and frame sequential imagingsignals B+BN, G, and R are sequentially output based on the accumulatedelectric charges. The series of operations are repeated while thesurface layer observation mode is set. The frame sequential imagingsignals B+BN, G, and R correspond to B2, G2, and R2 of the firstembodiment, respectively.

In the second embodiment, when generating a normal light image in themicrostructure observation mode and the microstructure and blood vesselobservation mode, a correction signal generation section 55 a in thenormal light image generation unit 55 generates a correction signal B′by removing the component of the blue narrow-band light BN from theframe sequential imaging signal B+BM, and generates a normal light imageusing the correction signal B′ and the frame sequential imaging signalsG and R. The correction signal generation section 55 a generates thecorrection signal B′ by multiplying the frame sequential imaging signalB+BN by a correction coefficient k that is determined in advance fromthe relationship between the amount of light of B color and the amountof blue narrow-band light BN or the like.

In the first and second embodiments described above, a pixel regionwhere the pixel value exceeds the threshold value Th1 of the brightnesssignal I is set as the superficial microstructure. Instead of this,however, a pixel region where the pixel value is within the highbrightness range N of the threshold values Th4 to Th5 of the highbrightness signal I may be set as the superficial microstructure. Theupper limit Th5 of the high brightness range N is set to be lower thanthe brightness value of halation caused by specular reflection ofillumination light. Therefore, it is possible to extract only thesuperficial microstructure instead of a halation region by performingextraction processing based on the high brightness range N. In addition,the threshold value Th1 is included in the high brightness range N.

What is claimed is:
 1. An endoscope system, comprising: an image signalacquisition unit for acquiring an image signal by imaging a subject; anormal light image generation unit that generates a normal light image,in which a wavelength component of a visible light region is included,based on the image signal; an irregularity image generation unit thatgenerates an irregularity image by extracting only information ofirregularities on the subject from the image signal; a first compositeimage generation unit that generates a first composite image bycombining the normal light image with the irregularity image; a bloodvessel extraction image generation unit that generates a blood vesselextraction image by extracting a blood vessel located at a specificdepth in the subject based on the image signal; a second composite imagegeneration unit that generates a second composite image by combining thenormal light image, the blood vessel extraction image, and theirregularity image; wherein the image signal is configured to includeimage signals of a plurality of colors including at least a blue signal,which has a blue light component having a degree of penetration to asurface layer of body tissue of the subject, and a green signal, whichhas a green light component having a degree of penetration to amedium-deep layer of the body tissue, and the blood vessel extractionimage generation unit extracts the blood vessel at the specific depthbased on a ratio of the blue signal and the green signal of the imagesignals of the plurality of colors.
 2. The endoscope system according toclaim 1, wherein the irregularity image generation unit includes amicrostructure image generation section that generates a microstructureimage as the irregularity image by extracting only a microstructure of abody tissue surface layer from the image signal, and the first compositeimage generation unit includes a normal light image and microstructureimage combination section that generates a first composite image bycombining the normal light image with the microstructure image.
 3. Theendoscope system according to claim 1, wherein the irregularity imagegeneration unit includes a hypertrophy image generation section thatgenerates a hypertrophy image as the irregularity image by extractingfrom the image signal only hypertrophy having a thickness from a bodytissue surface layer to a medium-deep layer, and the first compositeimage generation unit includes a normal light image and hypertrophyimage combination section that generates a first composite image bycombining the normal light image with the hypertrophy image.
 4. Theendoscope system according to claim 1, wherein frequency filtering of afrequency corresponding to a depth of a blood vessel is performed on theblood vessel extraction image.
 5. The endoscope system according toclaim 2, wherein frequency filtering of a frequency corresponding to adepth of a blood vessel is performed on the blood vessel extractionimage.
 6. The endoscope system according to claim 3, wherein frequencyfiltering of a frequency corresponding to a depth of a blood vessel isperformed on the blood vessel extraction image.
 7. The endoscope systemaccording to claim 1, wherein imaging of the subject is performed by acolor imaging device having pixels of a plurality of colors in whichrespective color separation filters are provided.
 8. The endoscopesystem according to claim 2, wherein imaging of the subject is performedby a color imaging device having pixels of a plurality of colors inwhich respective color separation filters are provided.
 9. The endoscopesystem according to claim 3, wherein imaging of the subject is performedby a color imaging device having pixels of a plurality of colors inwhich respective color separation filters are provided.
 10. Theendoscope system according to claim 4, wherein imaging of the subject isperformed by a color imaging device having pixels of a plurality ofcolors in which respective color separation filters are provided. 11.The endoscope system according to claim 1, further comprising: anillumination unit that sequentially irradiates the subject with lightbeams of a plurality of colors, wherein imaging of the subject isperformed by a monochrome imaging device whenever sequential irradiationis performed by the illumination unit.
 12. The endoscope systemaccording to claim 2, further comprising: an illumination unit thatsequentially irradiates the subject with light beams of a plurality ofcolors, wherein imaging of the subject is performed by a monochromeimaging device whenever sequential irradiation is performed by theillumination unit.
 13. The endoscope system according to claim 3,further comprising: and illumination unit that sequentially irradiatesthe subject with light beams of a plurality of colors, wherein imagingof the subject is performed by a monochrome imaging device wheneversequential irradiation is performed by the illumination unit.
 14. Theendoscope system according to claim 4, further comprising: andillumination unit that sequentially irradiates the subject with lightbeams of a plurality of colors, wherein imaging of the subject isperformed by a monochrome imaging device whenever sequential irradiationis performed by the illumination unit.
 15. The endoscope systemaccording to claim 1, further comprising: a display unit that displaysthe irregularity image.
 16. The endoscope system according to claim 2,further comprising: a display unit that displays the irregularity image.17. The endoscope system according to claim 3, further comprising: adisplay unit that displays the irregularity image.
 18. The endoscopesystem according to claim 4, further comprising: a display unit thatdisplays the irregularity image.
 19. An image generation method theendoscope system according to claim 1, comprising: acquiring an imagesignal by imaging a subject with an imaging device; generating a normallight image, in which a wavelength component of a visible light regionis included, based on the image signal using a normal light imagegeneration unit; generating an irregularity image by extracting onlyinformation of irregularities on the subject from the image signal usingirregularity image generation unit; generating a first composite imageby combining the normal light image with the irregularity image usingfirst composite image generation unit; generating blood vesselextraction image by extracting a blood vessel located at a specificdepth in the subject based on the image signal; generating a secondcomposite image by combining the normal light image, the blood vesselextraction image, and the irregularity image; wherein the image signalis configured to include image signals of a plurality of colorsincluding at least a blue signal, which has a blue light componenthaving a degree of penetration to a surface layer of body tissue of thesubject, and a green signal, which has a green light component having adegree of penetration to a medium-deep layer of the body tissue; andgenerating the blood vessel at the specific depth based on a ratio ofthe blue signal and the green signal of the image signals of theplurality of colors.